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Abstract 

This study focuses on evaluating students’ topic mastery and flow of thought in their Psycholinguistics project 
paper. Assessing these two aspects means a lot since it will be helpful as a planning tool to approach future better 
instruction. It was conducted with Indonesian students taking Psycholinguistics course at Humanities Faculty of 
UIN Maulana Malik Ibrahim Malang. The instrument used is scoring rubric of two sets to evaluate the two 
aspects; topic mastery and flow of thought. The finding shows that the students do not make the most of their 
writing skill to perform their topic mastery as they were lacking in using proper citation and rephrasing strategies. 
They need more practices to increase the quality of the project paper to perform a better critical thinking and 
originality. The flow of thought presented by the students in their project paper is influenced by the use of 
ineffective transition and lacking in applying writing format. However, presenting sufficient information details 
is not a problem to most of the students as the papers show the completeness of the information which can 
facilitate reader’s understanding. On language use, most students have fair competence in using correct grammar 
and appropriate vocabulary. 

Keywords: topic mastery, flow of thought, psycholinguistics 

1. Introduction 

Writing project papers becomes a complex skill for students as they need to identify a purpose, to produce and 
shape ideas and to refine expression to create an understanding of the targeted concept (White, 1995). 
Developing their understanding on content courses, for instance psycholinguistics, further shape their intellectual 
development (Johannesen, 2001; Bruning & Horn, 2000: 30). In this study, the learner’s understanding of the 
content course is assessed through student’s project paper reflecting their topic mastery and flow of thought. The 
assesment is made to monitor student’s progress, to evaluate how they need particular support and succeeding in 
the course (Student Assessment, 2011). 

Evaluating the student's progress in content courses can be done in several ways, for instance by 
monitoring the content mastery in common lecturing or given via online (Goldberger, 2011), or observing the 
patterns of the progress (Peha, 2003). Yet when the content course is given in foreign language the focus of the 
evaluation and assessment is in both the language proficiency as well as the content mastery (Weigle, 2002). In 
this case, the content mastery in project paper writing task should be assessed not only from the language used 
but also the fulfillment of successful ideas as the pattern to observe. 

In doing the assessment, the rater position should regard the purpose of assessing critical thinking. The 
rater is not ‘measuring’ writing as a scientist; rather, s/he is a humanist analyzing the thinking and reasoning- 
equally hermeneutic and rhetorical performances- of other human beings (Petruzzi, 2008). More interpretation 
toward the text is required to figure out what is actually happening in the learner’s thought. 

Student’s performance in project paper writing also can be assessed using the rubric which sets for the 
standard. The general standard involves eight items namely purpose, question, information, inference, concept, 
assumption, implication and point of view (Crook, 2006). Then, each student’s paper is scored by at least two 
trained raters using analytical rubric. Similar to any other assessment model the rubric is used as a helpful tool to 
measure the learners’ achievement in both project paper writing and content mastery. 

There are many other types of rubric as assessment aid. Analytic rubrics feature multiple scales that 
provide diagnostic information useful to both students and teachers. The criteria assessed in the rubric cover: (1) 
the investigative question explicitly stated; (2) a concise, accurate answer present;(3) samples from published 
research articles discussed; (4) confidence in conclusions discussed; and (5) overall quality of the statement 
(Connors, 2008). By using the provided criteria, the raters may avoid inclusion of unrelated factors, such as 
grammatical errors, and therefore it ensures consistent measurement of student’s writing performance. 

Despite the advantage of using rubrics as assessment aid, study results also indicated that using rubrics 
may not improve the reliability or validity of assessment if raters are not well trained on how to design and 
employ them effectively. Many teachers use rubric simply because they believe using any rubric is better than 
assessing without a rubric (Rezaei & Lovorn, 2010). Consequently, rubrics should be well-designed, topic- 
specific and complemented with exemplars to be more effective more. This means that rubrics should be 
developed for a specific purpose and a specific group of students. 

Basically, any assessment model makes strong connections with emerging conceptions of writing, 
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literacy and critical thinking suggesting an assessment approach in which writing is viewed as calling upon a 
broader construct than is usually tested in assessments that focus on relatively simple, on-demand writing tasks 
(Deane et al., 2008). Any model employed should be oriented to assessing not only the development of the 
student’s critical writing skills but also on the progress made by the students in term of content mastery, critical 
thinking and larger cognitive domain. 

In a nutshell, writing project paper should incorporate the teaching and learning which emphasizes on 
topic or content mastery and supports critical thinking skills reflected from the flow of thought. Education 
practitioners especially EFL teachers ought to view project paper writing task as a model developing student’s 
both writing skills and content mastery. Further, it entails the need of a well-designed and specific assessment 
model otherwise the learning objective of the course cannot be measured accurately. 

By figuring out the competence of English Department students in Psycholinguistics covering the topic 
mastery and flow of thought reflected in their paper, a better planning can be done to improve the quality of the 
course. Any difficulties of hindrance found will be a reference to plan in the way to assign students and create a 
better learning. As stated by Jackson (2009) for better topic mastery, it needs the teaching innovation such as 
breaking the learning goal down into smaller learning segments and more manageable chunks. In other words, 
chart the trajectory to achieving mastery of the goal of the course and identify checkpoints along the way. First, 
this study is to describe the student’s topic mastery on Psycholinguistics which is covered in the project paper. 
Second, the analysis done in this study aims at exploring how well the students express their flow of thought in 
writing project paper on Psycholinguistics issue. 

2. Methods 

The present study employs descriptive design by identifying the writing performance assessed in two aspects 
namely topic mastery and flow of thought. It uses qualitative approach as the aim is exploring the phenomenon 
of Psycholinguistics project paper writing of English department students which cover the pattern and dynamic 
of student’s topic mastery and flow of thought. 

This study was conducted with Indonesian undergraduate students taking English as their major. The 
participants were young adults with an average age of 20 and had intermediate level of English proficiency. The 
source of the data is the project paper written by the students taking Psycholinguistics course at Humanities 
Faculty of UIN Maulana Malik Ibrahim Malang. 

The instrument used is scoring rubric of two sets to evaluate the two aspects; topic mastery and flow of 
thought. Each of the rubrics is adapted from the available resources provided online for EFL teachers. The 
scoring process involves two raters in scoring the project paper to reach the objectivity of the evaluation result 
and to ensure the scoring consistency (Writing Assessment, 2000). Prior to the scoring process, the rubrics are 
discussed by the raters to make sure that the scoring result represents the student’s writing performance. 

As stated formerly the design of this research is quantitative therefore the question set is used as the 
main instrument. The data collection starts with collecting the project papers on Psycholinguistics and 
categorizing them based on the topic coverage. The data needed is in the form sentences in the paper showing 
the content mastery and flow of thought of the students. 

The first stage of the analysis is on putting the data into separate aspect based on the assessed skill. This 
process is needed to answer the first research question on the assessed topic mastery. On the skill to express 
oneself in written or the flow of thought, the data are analyzed based on the assessment rubric. 

3. Result and Discussion 

The topic mastery and flow of thought in the students’ project are analyzed separately in the following section. 
There were 172 papers analyzed as the data. 

3.1 Topic Mastery 

Topic or content mastery is examined through the elements of focus, clarity of purpose, development of idea 
organization, critical point of view and originality of writing, and the use of reference. These representation of 
textual information provides a measure of the degree of semantic relatedness indicating the extent to which the 
pieces of text are discussing the topic in the same way (Foltz et al., 2000). 

Concerning the focus, each student writes different topic. More students wrote on speech therapy (60%) 
compared to communication (35%) and only 5% on language therapy. Meanwhile on the focus, more students 
discuss adult language disorder (60%) compared to that in childhood (40%). 

The clarity of purpose of writing the project paper is assessed as it reflects the student’s ability to 
present their understanding by knowledgeable explanation. 45% of the papers show clear purpose in the 
introduction. Almost all of them clarify the significance of discussing the topic. While the rest do not mention 
the purpose of writing the paper. In addition, most paper of this group shows incoherent writing. They started 
the introduction with the definition of the disorder and the background of the disorder occurrence or the causes 
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of the disorder. Only some of them were aware of stating the purpose of the project paper as a standard academic 
rhetorical pattern. 

The development of idea organized in the paper also becomes an indicator of content mastery. It is 
expected that the students know how to breakdown the material from the general part into specific explanation. 
The prompt of the project has clarified the elements of the paper namely introduction, discussion and conclusion. 
On the idea development, the elaboration of the topic is not only based on the division but also organized into 
several subtopics. Concerning introduction, most of the students have one subtopic. Only some uses several sub 
topics as they divided the introduction into the research question and objective. Most of the paper show four 
subtopics in discussion namely the definition of the language disorder, the causes of the disorder, the 
characteristics of the disorder and the therapy for the disorder. 

The critical thinking and originality become another concern assessed in the project paper. The critical 
thinking is shown through the ability in presenting the writer’s point of view by effective use of citation. The 
paraphrasing technique is also seen as it clarifies the ideas showing the writing originality. The critical writing 
and originality is called evident when the student is able to present their point of view is not clear but the student 
still showed effort to rephrase the reference used. This aspect is called vague when the student’s idea is not 
observable as the writing is dominated by the reference compilation. 

On the ability for critical writing, the rhetorical pattern can be seen from the use of citation to support 
the topic presented in the paper. Only 8% show evident proposition and paraphrased well the supporting theory 
from credible resources. In addition, the originality of the writing is also an indicator observed. When the 
students can paraphrase correctly from the sources without relying too much on the wording in the reference, it 
shows their ability to elaborate the topic of the paper. Among the papers 43% show little reflection of critical 
thinking and originality. Students in this group have tried to paraphrase although insufficiently and use citation 
to some extent. Meanwhile, the majority (49%) do not paraphrase as they dominantly copy paste the reference. 
In addition, their citation and reference list are not matched. 

As the pattern of topic mastery, the data shows various results. Students who are able to express their 
content mastery by showing knowledgeable purpose of writing their project paper are also those who can use 
various references and are able to make good use of citation to show their critical thinking as well as originality. 
It occurs only on the topic Dysathria speech and ADHD communication. Although the topic is not easy, the 
students having good skill in writing can make the utmost of their writing skill to complete the project paper 
which can come to the expected learning goal. 

Despite the various topics chosen, the content mastery of the students shown by the knowledgeable 
elaboration of the topic in the project paper depends on their reading and writing performance. The finding has 
shown that the task of project paper writing is inseparable from reading references critically. The more 
references used, the better understanding of the topic becomes another concern. In order to write a good analysis 
and evaluation on a topic especially on content course like Psycholinguistics, careful reading of sources is 
essential. However, it needs more careful wording to make sure that the readers understand the purpose of the 
writing. In the data, only a little number of papers show the content mastery supported by clarity of stating the 
purpose of writing and present their critical thinking with their own effort. It is in line with Knott (2009) that the 
judgments and interpretations made based on the texts are the first steps towards formulating the writer’s own 
approach. The enlarging topic familiarity or the background knowledge of the topic is shown through the clear 
wording in the paper. 

On the other hand, students who are fail to present their knowledge and understanding of the topic will 
show a little or even no effort of developing their critical thinking as they do not show the originality of the ideas. 
They tend to copy paste the reference without thinking about the effect on readers. The compilation of the copy¬ 
pasting paragraph does not facilitate reader’s understanding. Furthermore, students in this group do not relate the 
citation and the reference list. The number of the reference does not support the content mastery. As an example, 
on topic Dyslalia, as the topic is not discussed in class, the student used a lot of reference. Unfortunately the 
references used do not support the content mastery as there are more copy pasted sentences compiled without the 
arrangement of ideas. The student has mentioned some sections in the paper but without sufficient paragraphing 
the readers would find it difficult to understand. It can be inferred that the writing only concerns with the 
quantity not the quality shown by the number of the sections, pages and reference used. 

Students who fail to express the purpose of project paper writing will also show little or no effort to 
perform their critical writing as well as put forward their original ideas (54%). Whereas, some students stay in 
“grey” area. It means that they are able to show evident clarity of purpose as they tried to write their introduction 
and conclusion part using their own critical point of view. Unfortunately, on discussion part they relied too much 
on the reference. A little effort made by paraphrasing some part of the reference but it still does not result in 
sufficient citation which facilitates reader’s understanding (29%). 

The rest (14%) have mentioned the purpose of writing their project paper clearly but the discussion 
parts are dominantly made up of the compilation of the paragraph from the reference used. Accordingly, the 
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level of originality decreases as the students do not clarify their own inference in using the references. 

As the data are taken from a Psycholinguistics in EFL context, the finding can be related to the other 
content courses requiring project paper writing. In the teaching of a content course assigning project paper 
writing, the student’s content mastery depends on several underlying factors which might either support either 
hinder the process. Based on the result of the above analysis, it can be inferred that the major factor is the writing 
proficiency of the students. It implies on difficulty to deal with the complex topics, the amount of references 
used and the ability to present critical writing. It goes with the underlying factors associated with foreign 
language acquisition as identified by Cenoz & Genesee (1998). 

By having the evaluation on the content of student’s paper as analyzed in this study, it can also be 
inferred that the method of course assignment needs more adjustment. As suggested by Boye (2012) the main 
strategies for better project paper writing involve prioritize and respond to content first. Prioritize means 
providing feedback on student’s writing based on the most important areas on which to concentrate. It is 
recommended to try focusing on "higher-order” concerns first, such as focus and development of the topic 
assigned. Therefore, it needs further implication to ensure that the students make improvement in their topic 
mastery through the project paper assignment. 

3.2 Flow of Thought 

Flow of thought becomes an aspect to evaluate in student’s project paper. It examines how students are able to 
recall and to use ideas which they did not have before. As the other aims, the evaluation is also done to see any 
improvement in student’s ways of thinking, to modify their reactions to aesthetic experiences as in the arts, and 
so on (Madaus & Syufflebeam, 1989). In this study, flow of thought is observed though some aspects presented 
in the project paper writing. They are the use of transition to facilitate reader’s understanding, the suitability with 
the academic writing format and the use of language. 

The use of transition characterizes the writer’s competence in guiding the readers to the expected track 
of idea flows. The transition words are observed either those existing within the paragraph or those bridging 
different paragraphs. The transition is called effective whenever it facilitates reader’s understanding. The finding 
shows more students show ineffective transition. 55 % had difficulty of using transition in writing their project 
paper. Almost all of them rarely used transition in the paragraphs or between the paragraphs. When this finding 
is confirmed with the data on citation, students in this group also have difficulty in paraphrasing. It means that 
the way the students of this group write is dominated by putting the citation altogether in their paper without 
considering how to arrange the ideas so that the flow of thought is well organized. On transition 45% show good 
use of transition. 

The students show lacking of the awareness of using the right writing format. As the finding shows 
twenty-two students ignore the importance of writing format, it represents 63% of the data that writing format 
does not belong to their concern. Students in this group mostly show imbalance paragraphing. Some paragraphs 
were written too long while some others were written in short paragraph. This way will affect the coherence of 
the writing. The longer the paragraph may influence the amount of main ideas which should be separated to 
facilitate reader’s understanding. This finding can be related to the finding on transition. As the students had 
imbalance or inconsistent paragraphing, they also ignore the need for transitional words to bridge the ideas 
between each paragraph. One student shows good paragraph format but he did not write conclusion. Even worse, 
there are two students having no paragraphing format at all. 

In general, presenting sufficient information details is not a problem to most of the students. 52% shows 
the completeness of the information which can facilitate reader’s understanding. 49% shows insufficient 
information details. Most students in this group failed to define some difficult terms and do not elaborate further 
detail concerning some terms used in their project paper. 

On language use, most students have fair competence in using correct grammar and appropriate 
vocabulary. 43% shows some errors in grammar and diction. However, the errors they made do not contribute 
much to the meaning. Meanwhile, 34% shows several grammatical errors. Even one paper confuses readers 
because of the poor language use. 23% performs good and effective language use. Although some papers show 
trivial mistake, the overall language use is still effective as it facilitates reader’s understanding. 

That the papers still concern on language use as inferred from the finding shows that students to some 
extent had conceptual preparation and planning. However, they still have not made make sure that the flow of 
thought in the paper is understandable for its readers. This is because each of them was busy with the paper 
completion so that they did not have peer checking before submitting their project paper. As noted by Treiman et 
al. (2003), ideal writers may have more need to do to reflect their flow of thought well, as the intended reader of 
a written text is often distant in time and space from the writer. This goal needs more time and assistance from 
the teacher for instance by prompting with detail instruction so that peer checking becomes student concern. 

The prompt not only to monitor the topic but also includes the flow of thought in paper writing. As 
suggested by Bowen (2010) outlining becomes the first step to organize a well thought paper. The finding of 
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this research is a starting point to reconstruct the instruction prompt so that the Psycholinguistics project paper 
quality can be improved. 

3.3 Relation between Topic Mastery and Flow of Thought 

The finding shows that the average topic mastery is 60 out of the highest possible score 100, while on flow of 
thought is 71. It means that the flow of thought is actually higher than the topic mastery. From this empirical 
data, it is obvious that the students have tried to implement their writing skill so that they can express their 
thought sufficiently. Meanwhile the topic mastery is lower because of the various topics chosen which is new for 
the students. Therefore, it is recommended that the range of topic should be put into a main concern for the next 
project instruction to improve the quality of topic mastery. 

To evaluate student’s both topic mastery mastery and flow of thought, holistic scoring or rating is used 
to ensure the representation of student’s performance thought the assigned writing task (Charney, 1984). Based 
on the scoring result students show different performance in topic mastery and flow of thought. 

The scoring of topic mastery involves the clarity of purpose and on critical thinking and originality. The 
average clarity of purpose is 30 out of the possible high score 40 which means that the ability to show the 
purpose of writing the paper is sufficient. The critical thinking average is 30 out of possible 60 meaning that it is 
still half way to the expected result. 

On flow of thought, the average score is 71 out of 100. The average effective transition is 15 out of 20 
as the highest score. It means the students mostly show sufficient use of transition. On writing format they got 14 
out of 20 meaning that they still need to have more concern on academic writing standard. The average score of 
information detail is 23 out of 30. It happens as the students explore their reference well to add the details of the 
issue presented in their paper. As the last aspect, the average language use is 19 out of the possible 30. It can be 
inferred that they need to improve a lot on the language use including grammar and diction. 

The relationship between the two scores is analyzed using correlation. The result of the correlation 
between topic mastery and flow of thought shows the coefficient of 0.5. It means positive and moderate 
correlation. This statistical finding gives evidence that the better use of flow of thought results in better topic 
mastery presented in the students project paper. 

4. Conclusion 

The student’s performances are various in terms of topic mastery and flow of thought. The flow of thought is 
actually higher than the topic mastery. The students have tried to implement their writing skill so that they can 
express their thought sufficiently. Meanwhile the topic mastery is lower because of the various topics chosen 
which is new for the students. The deficit of knowledge limits the level of argumentation or the depth of 
abstraction in exploring the issue. It implies on difficulty to deal with the complex topics, the amount of 
references used and the ability to present critical writing. 

On the topic mastery of, students still ignore the need to mention the purpose of writing the paper. In 
addition, most paper show incoherent writing due to their concern on the completeness of the project paper as a 
compilation of related references. It can be inferred that they do not make the most of their writing skill to 
perform their topic mastery on the Psycholinguistic issue presented in their paper. In addition, they need to 
improve their skill in using citation skill to facilitate reader’s understanding as there were many difficult terms 
left unexplained. Not only on citation but also rephrasing strategies need more practices to increase the quality 
of the project paper to perform a better critical thinking and originality. 

The flow of thought presented by the students in their project paper is influenced by applying transition, 
writing format, information details and language use. More students show ineffective transition which is 
confirmed with the difficulty in paraphrasing and using citation. It means that the way the students write is 
dominated by putting the relevant references altogether in their paper without considering how to arrange the 
ideas so that the flow of thought is well organized. The students also show lacking of the awareness of using the 
right writing format. In general, presenting sufficient information details is not a problem to most of the students 
as the papers show the completeness of the information which can facilitate reader’s understanding. On language 
use, most students have fair competence in using correct grammar and appropriate vocabulary. However, the 
errors they made do not contribute much to the meaning. 

The relationship between the topic mastery and flow though is in positive and moderate correlation. 
This statistical finding gives evidence that the better use of flow of thought results in better topic mastery 
presented in the students project paper. 
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